On the Capacity of the Binary-Symmetric 
Parallel-Relay Network 

Lawrence Ong, Sarah J. Johnson, and Christopher M. Kellett 



Abstract — We investigate the binary-symmetric parallel-relay 
network where there is one source, one destination, and multiple 
relays in parallel. We show that forwarding relays, where the 
relays merely transmit their received signals, achieve the capacity 
in two ways: with coded transmission at the source and a finite 
number of relays, or uncoded transmission at the source and a 
sufficiently large number of relays. On the other hand, decoding 
relays, where the relays decode the source message, re-encode, 
and forward it to the destination, achieve the capacity when the 
number of relays is small. In addition, we show that any coding 
scheme that requires decoding at any relay is suboptimal in large 
parallel-relay networks, where forwarding relays achieve strictly 
higher rates. 




Fig. 1. The binary- symmetric parallel-relay network with K relays 



I. Introduction 

We consider a class of parallel-relay networks where a 
source sends its data to a destination via many relays. From a 
theoretical point of view, analyses of parallel-relay networks 
are of great interest as they are embedded in more general 
multiterminal networks. The interest in using relays in a 
network is also driven by practical applications in which 
direct communication from a source to the destination is 
difficult. Parallel-relay networks model scenarios in which 
the destination obtains data from the source through spatial 
diversity (e.g., see macroscopic diversity |1|). An example is 
wired networks where there are multiple routes from the source 
to the destination. 

The network model we consider in this paper is the binary- 
symmetric parallel-relay (BSPR) network with K relays as 
depicted in Fig. [T| where each source-to-relay and relay-to- 
destination channel is a binary-symmetric channel. Our aim is 
to determine which, if any, transmission schemes can achieve 
capacity on the binary-symmetric parallel-relay network, and 
under what conditions. We investigate four possible transmis- 
sion schemes, which are the different combinations of whether 
the source and the relays send coded or uncoded signals, plus a 
hybrid scheme. We obtain capacity results when the network is 
small (in terms of the number of relays) using decoding relays, 
and when the network is medium to large using forwarding 
relays. The terms small, medium, and large will be made 
precise in the subsequent sections. 

Networks with relays were first introduced by van der 
Meulen |2| in which three nodes exchange data and any node 
can facilitate the message transfer between the other two. The 
special case where a source transmits data to a destination 
with the help of a relay, which itself has no private data 
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to send, was considered by Cover and El Gamal BJ, and it 
is now commonly known as the relay channel. Since then, 
many variants of relay channels have been investigated, for 
example: (a) the multiple-access relay channel ]4j, (5), (b) the 
broadcast relay channel [6], (c) the multiple-relay channel (7), 
(8), (d) the multi-way relay channel (9), (10), (e) the multiple- 
input multiple-output (MIMO) relay network |11| , (f) the 
multiple -source multiple-destination multiple-relay wireless 
network |12|-]14|, (g) the additive white Gaussian noise 
(AWGN) parallel-relay network (15)-(T8|, (h) the discrete- 
memoryless parallel-relay network with lossless channels 
from the relays to the destination |16) , and (i) the two-user 
BSPR network with lossless channels from the relays to the 
destination ]19| . To the best of our knowledge, the BSPR 
network with noise on both the source-to-relay and relay- 
to-destination channels has not previously been investigated. 
More specifically, this work differs from (h) and (i) in that the 
relay-to-destination channels are noisy and that the asymptotic 
behavior of the network when the number of relays grows is 
investigated. 

To date, the capacity of even the simple single-relay channel 
is not known except for a few special cases, e.g., the degraded 
case |3J, |7) (note that the parallel-relay network is not a 
degraded channel). This hints at the difficulty of analyzing 
multiterminal networks, especially large networks. 

Asymptotic capacity results, however, have been obtained for 
some AWGN networks. It has been shown that using coding at 
the source and no coding at the relays achieves the asymptotic 
capacity of the AWGN parallel-relay network [15| and the non- 
coherent MIMO relay network |11| as the number of relays 
increases to infinity. Here the source encodes the data and the 
relays merely scale their received signals and forward them. 
These results hold only for large AWGN networks (with the 
number of relays tending to infinity) and have not been proven 
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for other types of networks. In this paper, we will show a 
similar result for the BSPR networlQ i.e., no coding (only 
forwarding) at the relays achieves the capacity asymptotically 
as the number of relays tends to infinity. In addition, we will 
also show that using coded transmission at the source and 
no coding at the relays achieves transmission rates close to 
the capacity of the BSPR network with a generally medium 
number of relays, a result which has no current equivalent in the 
AWGN network. For example, for a network with cross-over 
probability of 0.2, we show that using 37 or more forwarding 
relays allows us to achieve within 0.0001 bits of the capacity. 

While it has also been shown [20 1 that uncoded transmission 
is optimal (in terms of the distortion measure) in the AWGN 
parallel-relay network where the source messages are Gaussian 
random variables, we show in this paper that in the binary- 
symmetric parallel-relay channel where the source messages 
are binary, uncoded transmission is strictly suboptimal when 
the number of relays is small. 

A. Main Results 

In this paper, we show that by applying coding at the source 
and using a finite number of forwarding relays, i.e., relays 
which forward their received signals without any decoding and 
re-encoding, performances arbitrarily close to the capacity of 
the BSPR network can be achieved. Alternatively, asymptotic 
(as the number of relays increases) capacity results hold even 
when both the source and the relays send uncoded message 
bits. For small networks on the other hand, forwarding relays 
do not achieve the capacity, but decoding relays, where the 
relays decode the source message, re-encode, and forward it 
to the destination, do achieve the capacity. Specifically, we 
have the following results, where the terms small, medium, 
and large used above have been made precise (recall that K is 
the number of relays in the BSPR network). 

• Theorems [T] and [2] give two upper bounds to the capacity 
of the BSPR network. 

• Theorems [3] [5] and [7] give three lower bounds (achievable 
rates) to the capacity of the BSPR network. 

• Corollary [6] gives the capacity of small BSPR networks, 
i.e., when 1 < K < K'(p s ,pd) for some K'(p s ,pd), 
defined in ( |22) , which is a function of the cross-over 
probabilities. 

• Theorems [4] and [9] give asymptotic capacity results of the 
BSPR network as K increases. 

• Corollary [5] gives achievable rates arbitrarily close to the 
capacity for medium to large networks, i.e., achievable 
rates within £ bits of the capacity for an arbitrarily small 
C > 0, when K > K(p, for some K(p 1 £), defined in 
GTJ, which is a function of the cross-over probabilities 
and £. 

II. Channel Model and Notation 

We consider the BSPR network depicted in Fig. [T] The 
network consists of a source (denoted by node S), K relays 

'The key difference (besides the channel alphabets) between the AWGN 
parallel-relay network and the BSPR network is that, for the latter the relays' 
signals are received orthogonally at the destination, see Fig. [T| 



(denoted by nodes 1,2..., K), a destination (denoted by node 
D), and a network of channels defined as follows: 

The channel from the source to the ith relay is a binary 
symmetric channel given by 

V t = U®Zi, (1) 

where U £ {0, 1} is the signal transmitted by the source, 
Vi £ {0,1} the signal received by the ith relay, Zj £ {0,1} 
is channel noise with Pr{Z; = 1} = p S) j, and © is modulo- 
two addition. The probability p S: i is also known as the cross- 
over probability for the binary symmetric channel from the 
source to relay i. Without loss of generality, we assume that 

< Ps , t < \- 

The channel from relay i to the destination is a binary 
symmetric channel given by 

Y i =X i ®E i , (2) 

where Xi £ {0, 1} is the signal transmitted by relay i, Yi £ 
{0, 1} is the signal received by the destination, and Ei £ {0, 1} 
is channel noise with Pr{Ei = 1} = pi Again, we assume 
that < p i4 < \. 

We assume that all the channels are independent, time 
invariant, and memory less. This means all Zj and Ei are 
independent of each other and are also independent and 
identically distributed over network uses (one network use 
is defined as simultaneous uses of all the channels in the 
network). 

For any channel variable A £ {U, Zi,Vi, Xi, Ei,Yi : 1 < 

1 < K}, we denote by A[t] the variable A at time t, i.e., A on 
the tth network use. We use a bold symbol to denote a vector 
(over time) of length n, i.e., A = (A[l], A[2], . . . , A[n]), where 
n is the code length. We will also use bar to denote a vector 
(over the relays) of length K, i.e., A = (Ai, A2, . . . , Ak), 
where K is the total number of relays. For a random variable 
A in upper case, the corresponding lower case a is used to 
denote the realization. 

A message W is observed by the source and is to be commu- 
nicated to the destination in n network uses. An (M, n) code 
is defined by the following encoding and decoding functions: 
(i) a message, W £ {0, 1, . . . , M — 1} = W; (ii) an encoding 
function at the source, U — fs(W); (iii) a set of n encoding 
functions at each relay, X^t] = /»,t(V^[l], Vi[2], . ..,Vi[t-l]), 
for all i £ {1, 2, . . . , K } and t £ {1,2, ... , n}\ (iv) a decoding 
function at the destination, W — g(Yi, Y2, ■ ■ . , Y k), where 
W is the estimate of W. Note that the relays' transmit signals 
can only depend on their respective past received signals. The 
rate of this code is (log 2 M)/n bits per network use. 

Let W be randomly and uniformly chosen from the source 
alphabet {0, 1, . . . , M — 1}. The average error probability is 

given by P e = i Y,w=o Pr {^ + W W = The rate R is 
achievable if the following is true: for any e > 0, there exists 
for sufficiently large n a (2 nR ,n) code such that P e < e. The 
capacity, C, is defined as the supremum of all achievable rates. 
We say that any node can decode the source message reliably 
if and only if the probability of erroneous decoding can be 
made arbitrarily small. 

A special case of the BSPR network is the symmetrical 
BSPR defined as follows: 
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Definition 1: A BSPR network is said to be symmetrical if 
p s ,i = p s and p iid = p d , for all i G {1, 2, . . . , if}. 
For the symmetrical BSPR, the channel from the source to 
each relay is equally noisy, and each channel from the relays 
to the destination is equally noisy. For the symmetrical case, 
we consider p s < 1/2 and p d < 1/2, as setting any of p s or 
Pd to 1/2, the capacity of the network is zero as the channels 
are randomized such that no information can be transferred 
from the source to the destination. 

III. Upper Bounds to Capacity 

In this section, we derive two upper bounds to C. Define 
H(p) = — plogp — (1 — p) log(l — p), for some < p < |, 
and H(0) = 0. We first have the following: 

Theorem 1: The capacity of the BSPR network is upper 
bounded by C < i? u b, where 

R uh 4 min ( max I{U; V),K-Y] H( Phd )\ . (3) 
I p{u) 7=1 J 

Proof: See Appendix [A] ■ 
We have the following observation: 

Lemma 1: The channel from U to V is symmetrical in the 
sense of (21] Thm. 4.5.2]. 

Proof: See Appendix [B] ■ 

Remark 1: Note the difference between a symmetrical 
channel in Lemma [T] and a symmetrical network as defined in 
Definition [T] For the rest of this paper, unless otherwise stated, 
the term symmetrical is used to refer to symmetrical BSPR 
networks. 

From Lemma [T] it follows that the maximizing input 
distribution for I{U ; V) is the uniform binary distribution, 
denoted by p'(u) J2l] Thm. 4.5.2]. 

The aforementioned bound, i? u b, has the same form as 
the cut-set bound [22 Thm. 15.10.1] except that the input 
distribution is the product distribution rather than the joint 
distribution. 

Noting that I(U; V) < H(U) < 1, we have the following 
(looser) upper bound, which is simpler and is independent of 

K: 

Theorem 2: The capacity of the BSPR network is upper 
bounded by C < 1. 

IV. An Equivalent Model for Forwarding Relays 

Before we present achievable rates of the different coding 
schemes, we first derive an equivalent model for the BSPR 
network when all relays do simple forwarding. 

Definition 2: Relay i, i G {1, 2, . . . , K}, is called a for- 
warding relay if it simply forwards its received signals, i.e., 



W 



U 



Y t = U® Ni 



Y 



D 



W 



X i [t] = V i [t-l], 



(4) 



for all t, and Xi[l] = by convention. 

Using only forwarding relays, the received signals at the 
destination can be re-written as 



EL = U®N l . 



(5) 



for all i G {1,2,..., K}, where iV; = Z { ® E t . We have 
dropped the time indices in the aforementioned equation as it 



Fig. 2. The equivalent point-to-point channel with forwarding relays, where 
Y = (Y 1 ,Y 2 ,...,Y K ) 



is clear that the destination receives the noisy version of the 
source's transmission two units of time later. It follows that 

Pi-{N. t = 1} = p s ,i(l - Pi,d) + (1 - p s ,i)Pi,d - Pi, (6) 

which is the cross-over probability of the effective binary 
symmetric channel from U to Yj when relay i is a forwarding 
relay. 

Thus, we have turned the network in Fig. [T] with only 
forwarding relays into a point-to-point channel from U to 
Y as depicted in Fig. |] Recall that Y = (Y u Y 2 , . . . , Y K ). 

V. Lower Bounds to Capacity 

In this section, we present four coding schemes and derive 
achievable rates for each scheme. 

A. Coded Transmission with Forwarding Relays 

We first consider coded transmission with forwarding relays, 
where (i) the source performs channel coding, and (ii) all the 
relays do simple forwarding. Therefore, we have an equivalent 
point-to-point channel from U to Y as described in the 
previous section. This coding scheme is also known as amplify- 
and-forward (8). Using this scheme, the following rates are 
achievable: 

Theorem 3: Consider the BSPR network. Coded transmis- 
sion with forwarding relays achieves any rate R < R co ded,f, 
where 

i?coded,f = I(U; Y) (7) 

evaluated with the uniform binary distribution p'(u). 

Proof: Using the results of the point-to-point channel [22 
Thm. 7.7.1], the aforementioned scheme achieves rates up to 
R < ^pj-I(U ; Y), as we use (n + 1) network uses in total to 
transmit an ni?-bit message. However, for any R that satisfies 
R < -Rcoded.f, we can always find a sufficiently large n such 
that R < ^pY^coded.f = ^iI(U;Y). Also, using a similar 
argument as the proof of Lemma [T| we can show that the 
channel U —> Y is symmetrical. So, the maximizing input 
distribution for I(U;Y) is the uniform distribution [21, Thm. 
4.5.2]. ■ 

Symmetrical case: p St i = p s and pi jd = p d , Vi 

Now, we evaluate Q for the symmetrical BSPR network: 
Corollary 1: Consider the symmetrical BSPR network. 

Coded transmission with forwarding relays achieves any rate 

R < Rcoded.t, where 



-R C0 ded,f = 1 + Kplogp + Kq log q 



K-l 

E 

z=o 



K-l 
I 



where p = p s (l 



Pd) 



■<- l P l )log(q l p K - l +q K - l p l ) 
(1 -p s )pd and q = 1 - p. 



(8) 
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Proof: Eqn. §8§ follows by evaluating the RHS of |7| with 
p s ,i = p s , Pi,d = Pd, Vi £ {1,2,..., A"}, and the uniform 
distribution p'(u). ■ 
Remark 2: Note that evaluation of the aforementioned rate 
has factorial time complexity in the number of relays, K. This 
also has implications for the evaluation of the capacity upper 
bound i? u [, in Theorem [T] 

General case: possibly different p Sii and p it d 

Now, we derive a lower bound for i? C oded,f that allows us to 
determine the performance of this scheme in the general BSPR 
network as the number of relays increases. First, we define 



Pmax(M) 



max p n 



(9) 



for some A4 = {mi, mj, . . . , ium} Q {1, 2, . . . , A"}. This 
means there exist M (out of K) relays where the effective 
channels from the source to the destination via these M relays 
are each not noisier than a binary-symmetric channel with cross- 
over probability Pmax(m)- We can show that the following 
rate is achievable. 

Corollary 2: Consider the BSPR network. Coded transmis- 
sion with forwarding relays achieves any rate R < i? co ded,f 
where 



R. 



coded,f 



M-l 

E 

;=o L 



> 1 



M 



MPMAX{M) logPMAX(A^) 
-M(l -_PMAX(A-l))log(l -PMAX(M)) 

V 



(Imax(m)Pmax(m) + 1max\m)Pmax(m)) 



M-l 



^°&(Qmax(m)Pmax(m ) 



Qmax(m)Pmax(m) ' 



(10) 



for any M C {1, 2, . . . , K}, where q M AX(M) = 1 - Pmax(m), 
and M = \M\. 

Proof of Corollary^ If a rate R is achievable using some 
M relays, then R is also achievable in the original network 
with K relays. Among these M relays, the maximum p nii is 
denoted by Pmax(m)- For each sub-channel from U to Y mi 
with cross-over probability p m . , we further add random noise 
to get Y^ . = Y m . © E'^ . , with some independent and random 
E'^ ( G {6, 1} where Pr{K H = 1} = PMA ^~ Pm > . In this 
modified network, we have 



l-2p„ 



YL = U®NL 



(11) 



where N' mi 



Z mi ® E mi © E'^ ( and Pr{iV^. = 1} = 
Pmax(M)' S A4. If the rate R' is achievable on the 

modified network from U to (Y^ .Y^ , . . . , Y^ ), then the 
rate R > R' is achievable in the original network using only 
M relays and is also achievable in the original network with K 
relays. Using the results from Corollary [1] we have Corollary [2] 

■ 

Remark 3: We will show in the next section (see RemarkfTTji 
that as long as we can find a sequence of M C {1, 2, . . . , K} 
(one for each K), such that |A1|(0.5 — Pmax(M)) — > cjo as 
K — > oo, then coded transmission with forwarding relays 
approaches the capacity. 



Remark 4: The condition in Remark |3] is not unreasonable 
as long as when the number of relays increases, the number of 
"bad" channels (with cross-over probabilities pi close to 0.5) can 
be kept at or below a certain fraction of K. Another example 
of a network satisfying the condition is one in which as K 
increases, the maximum cross-over probability is unchanged 
at some Pmax({i,2,...,k» =Pconst < 0.5. 

B. Uncoded Transmission with Forwarding Relays 

We now investigate the second scheme where the source and 
all the relays send uncoded signals. We first split the source 
message into n bits, i.e., W = (Wi, W2, ■ ■ ■ , W n ), where each 
Wi is uniformly distributed in {0, 1}. We use the following 
encoding functions: 

1) Uncoded transmission at the source: U[t] = Wt, 

2) Forwarding at the relays: Xi[t + 1] = Vi[t], 
for all t G {1,2,.. .,n}. 

Remark 5: In the aforementioned scheme, we send n bits 
in (n + 1) network uses. So the rate here is ^xj, which can 
be made arbitrarily close to 1 with a sufficiently large n. 

Remark 6: Note that the analysis in this section is slightly 
different from that in the previous section. In the previous 
section, we derived achievable rates as functions of network 
parameters. In this section, we fix the transmission rate at 
(arbitrarily close to) 1 bit/network use, which is a capacity 
upper bound, and analyze under what condition this rate is 
achievable. 

Symmetrical case: p sa = p s and p it d = Pd, Vi 

We have the following asymptotic capacity result as the 

network size increases using this scheme. 

Theorem 4: Consider the symmetrical BSPR network. For 

any e > and at any rate R < 1 = Auncoded.f, uncoded 

transmission with forwarding relays achieves P e < e with 

sufficiently large n and K. 

Proof: See Appendix [C] ■ 
From Theorem [2] we know that 1 bit/network use is an 

upper bound to the capacity. Hence uncoded transmission with 

forwarding relays achieves the asymptotic capacity as K tends 

to infinity. 

As the message is uncoded, the achievable rate is obtained 
not by typical-set decoding, but by maximum likelihood 
decoding where the destination simply decodes W = 1 if 
there are more l's than there are 0's in its received signals Y, 

General case: possibly different p s ti and p^d 

Now, consider the general case, and Pmax(-M) defined in |9| 
for some M. C {1,2,..., K}. We have the following corollary: 

Corollary 3: Consider the BSPR network. If there exists an 
M C {1, 2, . . . , K) for each K such that 



|M|(0.5 -Pmax(a-i)) 2 



as 



K 



00, 



(12) 



then for any e > and at any rate R < 1, uncoded transmission 
with forwarding relays achieves P e < e with sufficiently large 
n and K. 

Proof: See Appendix [TJ] ■ 
The observation in Remark [4] is also applicable to condition 
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The maximum likelihood decoding rule for the symmetrical 
case (i.e., by counting the number of l's in Y) might not 
be optimal for the general BSPR network. However, this 
suboptimal decoding rule is sufficient to show the asymptotic 
result in Corollary [3] 

The aforementioned results imply that with a sufficient 
number of relays, we do not require a channel code to achieve 
the capacity. This is because the relays provide sufficient spatial 
diversity for the destination to reliably determine the source 
message sent. 

C. Coded Transmission with Decoding Relays 

In the sub-sections above, we have derived achievable 
rates for the BSPR network with forwarding relays, and 
asymptotic capacity results have been obtained. In this section, 
we investigate a coding scheme using decoding relays, where 
the relays decode the source messages, re-encode, and forward 
them to the destination. This coding scheme is also known as 
decode-and-forward |8|. 

We obtain the following rate with decoding at some relays. 

Theorem 5: Consider the BSPR network. Coded transmis- 
sion with decoding relays achieves any rate R < i? co ded,d, 
where 

-Rcoded.d = min||l - H(p Stm .) : Vrn; e m|, 

M \ 

M -Y,H(p mi ,d)\, (13) 

for any M = {mi,m 2 , ■ ■ ■ ,m M } C {1,2, ... ,K} where 
M = \M\. 

Proof: See Appendix [FJ ■ 

Remark 7: The term M — Y^LiH{p mi ,d) = 
X)i=i max p(x m .) I{X mi ; Y m .) is the sum capacity of 
the individual point-to-point channels X„ H — > Y mi . The term 
1 — H(p S:rrii ) = max p ( u ) I(U ; V mi ) is the capacity from the 
source to relay mi. 

In the aforementioned coding scheme, only M out of the K 
relays are utilized, and the rest are not used. Although using 
more decoding relays increases the rate at which the destination 
can decode the source message [this can be seen from the term 
M — 53i=i H(Pmi.d) in JT5)L at the same time, it imposes 
an additional constraint 1 — H(p s ^ m .) in ( fO) for each relay 
rrii used, as each decoding relay must fully decode the source 
message. 

Although forwarding relays achieve the capacity asymptot- 
ically when K — > oo, decoding relays achieve the capacity 
under the following conditions: 

Theorem 6: Consider the BSPR network. Coded transmis- 
sion with decoding relays achieves the capacity under the 
following conditions: 

1) If K = 1, then 

C = l-ma X {H(p Stl ),H(p 1:d )} (14) 



2) If K > 1, and if K - Y% =1 H(jp jA ) < 1 - H(p s>i ) for 
all i £ {1,2, ... ,K}, then 

K 

C = K-Y,H{ Pl , d ). (15) 

i=l 

Proof: See Appendix [F] ■ 
Remark 8: The condition K > 1 in the aforementioned 
theorem corresponds to the case where that the sum of 
capacities of all K channels from the relays to the destination 
is smaller than the capacity of the channel from the source 
to each relay. This condition is likely to hold for small K 
and when the channels from the relays to the destination are 
noisy. This result resembles that of the single-relay channel 
where the decode-and-forward coding scheme outperforms 
other schemes when the source-to-relay channel is better than 
the relay-to-destination channel (8). 

Symmetrical case: p St i = p s and pi t d = Pd, Vz 
For the symmetrical case, we have the following: 
Corollary 4: Consider the symmetrical BSPR network. 

Coded transmission with decoding relays achieves any rate 

R < -R co ded,d, where 

i?coded,d = min{l - H(p s ),K(l - H( Pd ))}. (16) 

Proof: When p St i = p s , Vi, we have 1 — H(p s j) = 1 — 
H(p s ),Vi. So, selecting any M. will not affect min mie wi{l — 
H{Ps,m,i)} m < P~3] >- In addition, adding more relays into the set 
M. increases [M — Y^iLi as H{p mi ,d) < 1- Hence, 

it is always optimal to set M. = {1, 2, . . . , K}. So, ( [To} follows 
from ([13) with M = K. ■ 
Furthermore, whenp^ < p s , we have 1—H(pd) > l—H(p s ), 
and the achievable rate further simplifies to 

i?coded,d = 1 -H(p s ), (17) 

which is independent of K. 

D. A Hybrid Coding Scheme 

Using coded transmission and forwarding relays, noise on the 
source-to-relays channels propagates to the relays-to-destination 
channels. This can be rectified by having the relays decode the 
source message before forwarding it (hence removing the noise 
on the source-to-relay channels). Doing so, however, imposes 
additional rate constraints that the relays must fully decode the 
source message. We next propose a hybrid coding scheme in 
which some relays decode and re-encode the messages, and 
the rest of the relays forward their received signals. 

1 ) Achievable Rates: Denote the set of decoding relays by 
M. C {1,2,..., K} and the set of forwarding relays by T = 
{1, 2, . . . , K}\A4. Since all relays in the set A4 need to decode 
the messages, the rate is constrained by 1 — H (max ie7 vi P s ,i)- 
Each relay i E M fully decodes the source message and 
transmits using codewords Xi that are generated independent 
of U and of the codewords of other relays j e Ai. Under this 
scheme, we can view the BSPR network as a point-to-point 
MIMO channel from (U, X M ) to (F x , Y 2 , . . . , Y K ), where U 
and Xi, for all i £ A4, are statistically independent. This 
MIMO channel consists of sub-channels Xi — > Y.- L for all 
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Fig. 3. Comparing the hybrid coding scheme to the other coding schemes 



and the cut-set upper bound, K 
{1,2,. ..,8} 



Ps,i 



Pi,d 



0.3, for i e 



i € M (from the decoding relays to the destination), as well 
as U — > Xjr — > Yjr (from the source to the destination through 
forwarding relays). So, the destination can decode the source 
message at the rate /({/, X M \Y X ,Y 2 , Y K ) = I(U; Yjr) + 
I(X M ;Y M ) = I(U; Yjr) + J2 ieM I{ X i, Y i)- Hence < we have 
the following theorem. 

Theorem 7: Consider the BSPR network. The hybrid scheme 
achieves any rate R < i? co d e d,h. where 



R 



coded.h — 



1 — H I max p s i 
\ieM ' 



i£M 



(18) 



for any M C {1, 2, . . . , K} and T = {1,2,..., K}\ M, 
where H (max; e wt p S: i) = if A4 = 0. 

Remark 9: Setting all relays as forwarding relays, i.e., M. = 
and T = {1, 2, ... , K}, we recover Theorem[3] Setting only 
relays in.MC{l,2,..., K} as decoding relays and ignoring 
the other relays (i.e., not using them), i.e., T = 0, we recover 
Theorem [5] 

Determining the optimal set of M. is a hard combinatorial 
problem. A rule of thumb is to include relays with low p St i in 
A4, and relays with high p S: i in T . The reason is that a high 
p Si i (a noisy channel from the source to relay i) constrains the 
overall rate when i is included in M. (i.e., setting relay i to 
decode). 

2) Numerical Example: We now show that the hybrid 
scheme is useful when the channels from the source to 
the relays have different noise levels. As an example, we 
consider the eight-relay BSPR network with p s ^ — 0.1i/8, 
and pi t d — 0.3, for i £ {1,2, .. . ,8}. We compare i? co ded,f, 
R co d e d,d, and i? co d e d,h using different numbers of decoding 
relays. The results are depicted in Fig. [3] Setting all relays 
to perform forwarding, we achieve i? C oded,f = 0.52 (i.e., the 
hybrid scheme when \M\ — 0). With only decoding relays, 
the rate is maximized at i? C oded,d = 0.62 by using six relays, 
i.e., when six of the eight relays perform decoding and re- 
encoding and the rest of the relays are unused (note that 
using relay 8 as a decoding relay will constrain the rate to 



be i?coded,d < 1 — -ff (0.1) = 0.531). Using a combination of 
forwarding and decoding relays, we can achieve a significantly 
higher rate of i? co d e d,h = 0.71 by using four decoding relays 
and four forwarding relays. 

3) Other Combinations: In general, other rates can also be 
achieved using different coding schemes in which the relays 
perform other functions. Some examples are as follows: 

• The relays decode part (possibly different parts for 
different relays) of the source messages, re-encode, and 
forward them to the destination (demonstrated on AWGN 
parallel-relay networks in (23)). 

• The relays compress the received signals, and forward 
them to the destination (demonstrated on AWGN parallel- 
relay networks in [24 1). 

• A combination of different schemes at different relays 
(demonstrated on AWGN parallel-relay networks in [25]). 

4) An upper bound to achievable rates with decoding relays: 
For any coding scheme, if any of the relays is required to 
reliably decode the source messages]^] the achievable rate is 
necessarily constrained by the following: 

Theorem 8: For any (2 nR ,n) code, if any relay i £ 
{1, ...,K} is to reliably decode the source message, then 
the rates achievable by this code are upper bounded as 



R < max I(U;Vi) 

p(u) 

= 1 - H(p s>i ). 



(19a) 
(19b) 



Proof: Since we need to ensure that the probability of 
relay i wrongly decoding the source message can be made 
arbitrarily small, the rate R must be upper bounded by the 
channel capacity from the source to relay i. From \22 Thm. 
7.7.1], we have ( |19a| >. Since the channel from U to Vi is a 
binary symmetric channel, we have ( |19b| i. ■ 

VI. Capacity vs. Number of Relays 

In this section, we consider the symmetrical BSPR network. 
By varying the number of relays, K, we analyze the perfor- 
mance of the three schemes considered in this paper: (i) coded 
transmission with forwarding relays, (ii) uncoded transmission 
with forwarding relays, and (iii) coded transmission with 
decoding relays. Note that the hybrid scheme is not used 
here as we will either use all relays as decoding relays or 



as forwarding relays (see Remark 10 below). Recall that 

P = Ps{l - Pd) + (1 - Ps)Pd- 

Remark 10: Note that the hybrid scheme consisting of some 
combination of decoding and forwarding relays is not useful in 
the symmetrical BSPR network. From TheoremlT] i? C oded,h = 
min {l - H( Ps ), I(U; Yjf) + \M\(l- H(p d )^ [if M + 0, 

and -R co ded ; h = I(U; Yri.a, ...,k}) if -M = 0, where T is the 
set of forwarding relays and M. is the set of decoding relays. 
From the data-processing inequality, we know that I(U; Yjr) < 
I(Xjr; Yjr) = - H(p d )). So, if M ^ 0, we will choose 
A4 = {1,2,..., K}. This means we either use all relays as 
forwarding relays or as decoding relays. 

2 This condition is not necessary for reliable communication because reliable 
decoding at the destination does not necessarily require reliable decoding at 
any relay. 
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TABLE I 

Number of relays sufficient to achieve certain error 
probabilities using uncoded transmission 



1000 



K(p,0 



Fig. 4. Gap from the 1-bit capacity upper bound, £, vs. K(p, Q for different 

V 



A. Coded Transmission with Forwarding Relays 

We first show the following asymptotic result as K tends to 
infinity using coded transmission with forwarding relays. 

Theorem 9: Consider the symmetrical BSPR network. 
Coded transmissions with forwarding relays achieves any rate 
R < i? C oded,f as defined in (|7J, where 

-Rcoded,f C, as K oo. (20) 

Proof: See Appendix |G| ■ 
Remark 11: We can use the aforementioned result to prove 
our claim in Remark [3] Even for a general BSPR network 
(which can be non-symmetrical), coded transmission with 
forwarding relays approaches the capacity asymptotically as the 
network size increases if the condition specified in Remark [3] is 
satisfied, i.e., |.M|(0.5 — Pmax(a-i)) ~~ ► 00 as K ~> °°> where 
M C {1,2,..., K}. From < [il)} , we note that i? code d,f > R' 
where R' equals the rate achievable by the same coding strategy 
in the symmetrical BSPR network with K replaced by \M\, and 
P by Pmax(m) ( c -f- {§})• F rom I H-Od) , we know that R' — » C 
as A" — > oo. It follows that i? C oded,f — > C as K — > oo for the 
general BSPR network. 

Theorem [9] implies that the capacity rounded to some number 
of significant digits is achievable with coded transmission and 
K(p, £) or more forwarding relays, for some positive integer 
K(p, C) given in the following corollary. 

Corollary 5: Consider the symmetrical BSPR network. For 
any £ > 0, if K > K(p, £), where K (p, Q is the smallest 
integer satisfying 



1 + K(p,()plogp + K(p,()qlogq 
K( P ,0-i 

£ 



1=0 



log(q l p K ^0-l + qKM-ljf) 



>1-C, (21) 



then coded transmission with forwarding relays achieves rates 
within £ bits of the capacity. 

Proof: Because I(U; Y u Y 2 , . . . , Y K+1 ) > /(C/;y l5 F 2 , 
. . . , Yk), i? C oded,f is a non-decreasing function of K. Note that 
the LHS of ( f2"Tj ) is i? C oded,f (see Corollary [TJ. From ( |40e) > we 
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know that there must exist a positive integer K(p, £) such that 
pT) is true. Since C < 1 from Theorem [3] we have i? co ded,f > 
C - C for A = K(p, C) as well as for all K > K(p, (). ' ■ 

As K(p,Q in Corollary |5] is not available in closed form, 
we numerically evaluate K(p,() for p E {0.1,0.2,0.3,0.4} 
and for varying £. The results are shown in Fig. |4] At p = 0.1, 
we only need 16 relays to achieve within 0.0001 bits of the 
capacity. At p — 0.4, we need 387 relays to achieve within 
0.0001 bits of the capacity. 

The proof of i? co( jed,f relies on the channel coding theorem 
which requires an infinitely long codelength n. This means we 
have an infinitely long delay between the time a message is 
transmitted at the source and the time the message is decoded 
at the destination. 

B. Uncoded Transmission with Forwarding Relays 

In Theorem [4] we have shown that when transmitting at 
1 bit/network use using uncoded transmission with forwarding 
relays, an infinitely large number of relays are required to 
achieve an arbitrarily small error probability. However, if a 
larger error probability is acceptable, the number of relays 
required is smaller. The proof of Theorem [4] provides an 
inequality explicitly upper bounding the error probability by a 
function of the number of relays and the crossover probability 
[see ( |3l~j ) in Appendix |C) . Table [I] shows the number of relays 
that are sufficient to achieve P e up for different channel cross-over 
probabilities p G {0.1,0.2,0.3,0.4}, where P" p is an upper 
bound to the probability of decoding error at the destination, 
P e . 

C. Coded Transmission with Decoding Relays 

We have seen that forwarding relays are asymptotically 
optimal for large K. Now, we show that coded transmission 
with decoding relays achieves the capacity when K is smaller 
than a certain positive integer which depends on the cross-over 
probabilities. From Theorem [6] we have the following result. 

Corollary 6: Consider the symmetrical BSPR network. If 
K < K'(p si p d ), where 

' 1 - H{p s ) X 
. '1-H(p d )! } 

then coded transmission with decoding relays achieves the 
capacity. 

The proof of i? C oded,d again relies on the channel coding 
theorem. Hence, we also have an infinitely long delay for 
coded transmission with decoding relays. 

Remark 12: From Theorem [8] we know that for any coding 
scheme in which one or more relays are to decode the source 



K'(p s ,p d ) 



(22) 
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TABLE II 

Conditions under which various coding schemes achieve the capacity 



Coding schemes 


Delay 


Network size (K) 


Coded transmission with forwarding relays 
Uncoded transmission with forwarding relays 
Coded transmission with decoding relays 


Infinitely long 
Two network uses 
Infinitely long 


K>K(p,Q 
Infinitely large 

K<K'( Ps , Pd ) 




message, rates at most -Rdecode = 1 — H{Ps) can be achieved, 
which is independent of K and is bounded away from the 
capacity upper bound of 1 bit/network use, for any p s > 0. 
Since we have shown that as K increases, rates arbitrarily close 
to 1 bit/channel are achievable, it follows that there exists a 
positive integer K' where for any K > K', we have that 
^decode < ^?coded,f < C, i.e., any coding scheme that requires 
decoding of the source message at any relay is suboptimal in a 
large BSPR network — in such cases, forwarding relays achieve 
strictly higher rates. 

D. Summary 

From the above subsections, the three coding schemes 
achieve the capacity of the BSPR network under different 
conditions as summarized in Table UTJ 

E. Numerical Examples 

We present two numerical examples to compare achievable 
rates of coded transmission with forwarding relays and decod- 
ing relays to the capacity upper bound with varying K and 
with the following parameter values: (i) p s = pd = 0.1 and (ii) 
p s = 0.05, pd — 0.3. In the first network, the source-to-relay 
and the relay-to-destination channels are equally noisy, while 
in the second network, the source-to-relay channels are less 
noisy. 

~ We 



The results are shown in Figs. 5a and 5b 



see that 

decoding relays achieve the capacity when K is small, i.e., 
K = 1 when p s = Pd = 0.1, and K < 6 when p s = 0.05 and 
Pd = 0.3. We can see from ( [T6] l that when K > jz?gfey> 
the maximum achievable rate of decoding relays is fixed 
at i? C oded.d = 1 — H(p s ). Hence, using decoding relays 



is suboptimal when the number of relays is large (where 
forwarding relays achieve close to 1 bit/network use). Using 
forwarding relays, as predicted, approaches the capacity upper 
bound as the number of relays increases. 

VII. Reflection 

We have investigated the binary-symmetric parallel-relay 
network. We derived achievable rates using different coding 
schemes that utilize forwarding or decoding relays or a mix 
thereof, as well as coded or uncoded transmission at the source. 
We have also analyzed the network as the number of relays 
grows to infinity. 

With coded transmission, forwarding relays achieve the 
capacity (rounded to some number of significant figures) for 
networks with a finite number of relays. For instance, for cross- 
over probability p = 0.1, we need 16 relays or more to achieve 
within 0.0001 bits of the capacity, and for p — 0.4, we need 
387 relays or more to achieve the same result. However, an 
infinitely long code length, n, is required to drive the error 
probability to zero. Decoding is done after the destination 
receives n channel outputs over time which necessarily incurs 
a large delay. 

With coded transmission, decoding relays achieve the capac- 
ity of networks with one relay, and networks with more relays 
if the sum of capacities of all the channels from the relays 
to the destination is smaller than the capacity of the channel 
from the source to each relay. Again, decoding is done after 
the destination receives n channel outputs over time, which 
necessarily incurs a large delay. 

With uncoded transmission, decoding is almost "instan- 
taneous", i.e., when the destination receives the noisy bit 
transmitted by the source. The transmission from the source 
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p*(10001 ■ ■ • 01|0) p* (01110 • • ■ 10 0) 
p*(10001 • ■ • 01| 1) p* (OHIO • ■ ■ 10 1) 



(23a) 
(23b) 



to the relays and the transmissions from the relays to the 
destination take two network uses, and this is the total delay 
incurred. With uncoded transmission, message bits are sent at 
1 bits/network use which is an upper bound to the capacity. 
However, to drive the error probability to zero, an infinitely 
large number of relays is required. 

Coded transmission with decoding relays (which removes 
the noise on the source-to-relays channels) performs well when 
the number of relays is small because the number of additional 
rate constraints — required since each relay must decode the 
source message — is also small. These additional rate constraints 
actually limit its performance when the number of relays 
increases. Using forwarding relays, even though the noise on the 
source-to-relays channels propagates to the relays-to-destination 
channels, the increase in the number of relays provides the 
destination with sufficient spatial diversity to decode the source 
message. We can view both coded and uncoded transmission 
schemes with forwarding relays as "spatial" repetition codes, 
where the minimum Hamming distance of the code increases 
with the codelength, and the codelength increases with the 
number of relays. 

The aforementioned observations lead to the design of the 
hybrid scheme, where we use some relays as forwarding relays 
and the others as decoding relays. More specifically, we use 
relays with better source-to-relay channels as decoding relays, 
and relays with noisy source-to-relay channels as forwarding 
relays. The reason for this choice is that if a relay is able to 
decode the source message without constraining the overall 
transmission rate (which is likely when the channel from the 
source to this relay is good), we should let the relay decode 
the source message on that link to stop noise propagation. This 
scheme can improve the performance over purely forwarding 
relays and purely decoding relays in non-symmetrical BSPR 
networks where the source-to-relay channels are not all equally 
noisy. 



Appendix A 
Proof of TheoremQ] 

First, we note that W -> U -> V -> X - 



Y -> W forms 

a Markov chain. Using Fano's inequality (22] Lem. 7.9.1], we 
have that H(W\W) < 1 + P e nR. So, 



nR = H(W) = H(W\W) + I(W; W) 
< 1 + P e nR + I(U;Y) 
R < I(U;V)+P e R+l/n 



(24a) 
(24b) 
(24c) 



where ( |24b[ ) follows from Fano's inequality and by applying 
the data processing inequality (DPI) [22, Thm. 2.8.1] to the 
aforementioned Markov chain, and ( |24c[ ) follows by applying 
DPI again and because the channel from U to V is memoryless. 



We also have that 

I(U;Y) <I{X;Y) <nI(X;Y) (25a) 

K K 

= n^7(X i ;y i )<n^[l-JJ(p M )] (25b) 

i=l i=l 

where ( |25a[ > follows from the DPI and because the channels 
from X to Y are memoryless, and ( |25b| > is derived because the 
uniform distribution p'(xi) maximizes the mutual information. 
From |24b) and ( |25b| i, we have 

K 

R<K-J2 H (Pi,d) + P,R + 1/n. (26) 



Setting ij->oo and then P e —> for conditions ( |24c| i and 
26}, we have Theorem [T] ■ 



Appendix B 
Proof of LemmaQ] 

We write the matrix of transition probabilities of the channel 
as ( |23a| i. The top row is the conditional probability of V given 
U = and the bottom row {7 = 1. We rearrange the columns 
and pair up columns p*(v\u) and p*(v\u) as a sub-matrix, 
where v is v with all the bits flipped. Each sub-matrix is boxed 
in d23fj) . Clearly, p*(v\u') = p*(v\u") if u' ^ u" . In each sub- 
matrix, the top row is a permutation of the bottom row, and 
the left column is a permutation of the right column. Hence, 
the channel is symmetric in the sense of pT[ page 94]. ■ 

Appendix C 
Proof of Theorem|4] 

As forwarding relays are used, we have the equivalent 
channel in Fig. [2] with U = W . At the destination, the 
received signals are Yi — W ® Ni, where JVj = Zi © Ei 
and Pr{Ni = 1} = p as defined in Q. 

Let y = (j/i, 2/2, • ■ • , Uk ) be the received signals at the des- 
tination. The optimal decision decoding rule, which minimizes 
the error probability, is: 

. (0, if Pr{W = 0\Y = y} >Fr{W = l\Y = y} 
ll, otherwise 

(27) 

Applying Bayes' rule to |27} and noting that Pr{W^ = 0} = 
Pr{M^ = 1} = h, the condition for w = in |27} becomes 

Pr{? = y\W = 0} > Pr{F = y\W = 1}. (28) 

This is known as the maximum likelihood decoder. Since 
Pi — p, Vi E {1, 2, . . . , K}. The decision rule in ( f2"8"j ) becomes 

(1 - p )0(syr-o(s) > (1 _ p )*-o(sy (fl (29) 
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where 0(y) is the number of 0's in y. As < p < |, we have 
the optimal decoding function at the decoder as follows. 



W = g(Y) 



'0, if 0(F) >f 
1 , otherwise 



(30) 



Having defined the encoding and decoding functions for 
uncoded transmissions with forwarding relays, we derive the 
error probability. 

P e = Pr{W = 1\W = 0}Pr{W = 0} 
+ p T {W = 0\W = 1} Pt{W = 1} 



If- A 
2 Pr 0(F) <- 



1 



2 Pr 0(F) > 2 



W = 
A 



W = 1 



= Pr <^ 0(F) > 



A' 



vt/ = 1 } = Pr |E iV ^y} 



= Pr 



Jit*- 



< 



exp 



-2A 



P>~ 2 -P 



(31) 



where ( fJT) is due to Hoeffding 1 26 Thm. 2] if | — p > 0. 

By sending uncoded bits at the rate ^xj bit/network use, 
we know from ( f3T| that the error probability can be bounded 
by 

P e < cxp (-K6) , (32) 

where S = 2 ( \ - p) 2 > 0. So, for any < p < \ and e > 0, 
we can select A > \ In - such that P e < e. ■ 



Appendix D 
Proof of Corollary[3] 

We use the idea in the proof of Corollary [2j i.e., using only 
M relays for some M < K. If a coding scheme achieves 
P e < e using only M relays, then it can also achieve P e < e 
with A relays. Again, for each sub-channel from U to F m . 
with cross-over probability p mj , we further add random noise 
to get F^. = Y mi © A^, where £^ € {0, 1} and Pr{£^. = 
1} = PuA i^l~ Pmi ■ Now ' each sub-channel from [/ to F^. is a 
binary symmetric channel with cross-over probability Pmax(a^)- 
We use the following decoding rule: 



Appendix E 
Proof of Theorem|5] 

We use the following super-block coding scheme. Con- 
sider B blocks each consisting of n network uses. We 
split the source message into (B — 1) equal parts, i.e., 
W = (Wi, W2, ■ ■ ■ , Wb-i) where each Wi is independent 
and uniformly distributed in {l,...,2 nR }. In each block 
b G {1,2, . . . ,B-1}, the source transmit U^ b \W b ). Denoting 
the received symbols of relay i in block b by v[ , relay i 
transmits xf^iV^) in block (b+ 1). The destination then 
decode W b from its received symbols in block (b + 1), i.e., 

In the following, we will only consider the transmissions 
from the source to the relay in block 1, and those from the 
relay to the destination in block 2. Suppose that the destination 
can reliably (with arbitrary low error probability) decode W\. 
Repeating the same transmission scheme for the source-relay 
channel in block b and for the relay-destination channel in 
block (6 + 1) for all b € {2, 3, . . . , B - 1}, the rate of R' = 
(B -~ 1)R/B is achievable. This means we can achieve rate 
R' — > R by letting B — > 00. Where appropriate, we drop the 
superscript that indicates the block to simplify notation. 

We first select a set of M relays, and let the set of selected 
relays be {TO 1; m 2 , . . . ,m^/} = M. In block 1, the source 
sends U(Wi). Choosing p'(u) to be the uniform distribution, 
if 

R <I(U; V mi ) = 1 - H(p Simi ), (35) 

for all m,i £ A4, then each relay in A4 can reliably (with 
arbitrarily low error probability when n is sufficiently large) 
decode W\. 

In block 2, the relays in A4 transmit the decoded W\ to the 
destination. The rest of the relays transmit zero, Xj = for all 
j M. Choosing p(x mi , x m2 , . . . , x mM ) = JJiLiP' {x mi ), 
where each p'(x mi ) is the uniform distribution, if 



R<I{X mi , 



■ ; X rriM , Y mi , . 

M 



Y 



(36a) 



= V/(X m< ;F mi ) = Y\l-H(p iA )l (36b) 



i=l 



then the destination can reliably decode W\. Note that X mi 
are independently generated. So, \36b\ follows because F TOi 
is independent of {Y nlj : j ^ i}, and Y mi is also independent 
of {X m . : j ^i} given X m .. 

If a rate R satisfies (J35j for all e M and ( |36b| i, then 
the destination can reliably decode W\. Repeating this scheme 
for all blocks, we have Theorem [5] ■ 



W 



if 0(F' Y' Y' ) > 



(33) 



1, otherwise 

From ( (3T) , we know that the error probability of this decoder 



Pe < exp 



(34) 



As A — > 00, if M (i — Pmax{m)) ~ * °°> m en for any e > 0, 
we can find some A4 such that P e < e. ■ 



Appendix F 
Proof of Theorem|6] 

Compare P u b in Theorem [T| and P C oded.d in Theorem |5] For 

A = 1, we set M = 1, and we have max p ( u ) I(U; V%) = 
1 - H{p St \). So, Pcoded.d = Pub, and is the capacity. 

For A > 1, setting M = A, if A - Y,f=i H (Pj,d) < 
1 — H(p s i ) for all i £ {1, 2, ... , A}, then P co d e d,d reduces to 
(B). Since I(U; V x , V 2 • . . , V K ) > I(U; V,) for all i, we also 
have max p ( u) I(U; V) > K - J2f=i H iPj,d)^ ^ ^ub also 
reduces to (15) . ■ 
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Appendix G 
Proof of Theorem|9] 
We first assume that U = is sent. We define the 77-strongly 
typical set, denoted by TryM^ with respect to the distribution 
p*(y\u = 0) as a set of vectors {y} such that 



K 



N(l;y)-p 



K 



N(0;y)-q 



(37) 



where N(a; y) is the number of occurrences of the symbol 



a e {0, 1} in the sequence y. See |27 page 73] for a more 



general definition of strongly typical sets. 

We assume that < p < | is a rational number and consider 
a sufficiently large integer K' such that K'p is an integer. For 
any K' and p, we can choose 77 sufficiently small such that 
"T~lY\o]ri ~ { all y each having K'p number of l's and K'q 
number of 0's}. This means, for each y £ Tryim^, we have 
p*(y\0) — p K ' p q K ' q . By the same argument, T^L^ = {all y 
each having K'p number of 0's and K'q number of l's }, and 
hence p*(y\l) = p K ' p q K ' q . 

Recall that the input distribution p'(u) is the uniform 
distribution. Suppose that some y has K'p number of 0's 
and K'q number of l's. We have p*(y|0) = q K p p K q and 
p*{y\l) = q K ' q p K 'P. So, 

p*(y) = Pr{[7 = 0}p*(y (l) \0) + Pr{U = l}j>*(y (0 |l) 

f' q p K ' p )± a {K',p). (38) 



= kq K ' P P K ' q 



Since the effective noise, 2Vj, in each channel U — > Yi is inde- 
pendent and = 1} is the same for all i e {1,2,..., K}, 
Y.- L are i.i.d. given U [from Q]. So, we have, for K' sufficiently 
large, [27 Theorem 5.2] 

Pr{Y e T$[ U]V \U} > 1 - r,. (39) 

Now, for some sufficiently large K', 

H(Y) = -J2p*(y)logp*(y) 

v 

= - E P*(y)\ogp*(y) 

all y that have K' p number of 0's 

E P*(y)^ogp*{y) 

all y that have K 1 q number of 0's 

all y that do not have K p or K q number of 0's 

>- Y p*(y) lo &p*(y) - Y p*(y) l °gp*(y) 

= - E \\p*(y\ i )+p*im]\ogp*{y) 

E \[p*m)+p*{y\m°zf{y) 



yeT, 



y|oj>7 



>- E \p* (.y\ l ) lo %p* (y) 

~ E 9P*(yi°) lo gp*(y) 



(S) 



SETT: 



J log a(K',p) E 



V|0]t, 



~toga(K' , P )Pt{Y €T$ 1]n \U 



\oga(K',p) Pr{F G 7f 



K 

[Y\0] V 



u 



1} 

0} 



> 



loga(X',p)(l-7 ? ) 
loga^'jp) - 77', 



where 77' = — r]\oga(K' ,p) > which can be chosen 
arbitrarily small by choosing an arbitrarily small 77 for any K' 
and p. Note that the equality (jj) follows from ( |3~8| ). 
Now, 



i?coded,f = J(17; = H(Y) - H(Y\U) 
> - log a(K',p) - 77' - if'7f(p) 
= - loga(K',p) + K'(p\ogp + qlogq) 
I 



log 



1 



V 

1-p 



K'(l-2p) 



V 



1 = R 



ub • 



as K' 



00, 



(40a) 
(40b) 
(40c) 

(40d) 

(40e) 



yeT, 



where 77' can be chosen arbitrarily small for any K' and p, 
and 1 bit/network use is an upper bound to the capacity. 
Note that ^ < 1, and 1 - 2p > 0. The above result 
holds for large K' where K'p is an integer, But since 
I(U;Y 1 ...,Y K+1 ) > I(U;Yx,...,Y K ) for all positive in- 
tegers K, we have Theorem [9] ■ 
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